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ABSTRACT 

Subjects for an investigation of the effects of 
training with a computer and past computer experience on the 
computerized aptitude test performance of college students were 92 
undergraduates at the University of North Carolina at Charlotte, who 
participated in a test study utilizing six Apple 11+ microcomputers. 
The study was conducted to determine whether: (1) computerized test 
performance would be significantly enhanced and the concomitant 
anxiety level would be significantly lowered by increasing the amount 
of training with the computer prior to testing; and (2) increasing 
the amount of training with the computer would significantly reduce 
the difference between the mean anxiety level occurring with a 
computerized test and the mean anxiety level occurring with a 
paper-and-pencil test. Test performance was measured by the 
number-correct score on an arithmetic reasoning test. Results 
indicate that neither training immediately before testing nor past 
computer experience appear to significantly affect computerized test 
performance or the concomitant anxiety level. The mean 
paper-and-pencil test score was higher than the mean computerized 
score, however, and it is suggested that differences between 
pancil-and-paper and computerized test performance may be the result 
of human error and deficiencies of the computer software. A list of 
references completes the document. (JB) 
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(1) The effects of training on computerized aptitude test 
performance and anxiety 

(2) Topical Session 

a . General- -Organ i za tiona 1- Indus tria 1 

b . \ddi tiona 1- -Other : Measurement 

c. Cognitive Process — Other : Measurement 

(3) Purpose 

Despite the advantages of computerized testing (see Burke & 
Normand, 1984; Space, 1981), the completion of a computerized 
test may be more difficult for some and easier for others. The 
effects, if any, of computerized testing procedures on examinees' 
performance are not clear. The relatively few studies which have 
been conducted have provided mixed results. completion of 
different arithmetic reasoning tests by computer has been found 
to enhance (Johnson &Mihal, 1973), hinder (Lee, Moreno, & 
Svmpsoh, and not affect performance (Wildgrube, 1982). 

Lee et al. (1984) suggested more research to identify factors 
which may affect performance on computerized tests. The 
literature (see Hansen & 0 ! Neil, 1970; Hedl, O'Neil, & Hansen, 
1973; Johnson & White, 1980; Johnson & Johnson, 1981) suggests 
that interacting with a computer may evoke a signiticant amount 
of anxiety to affect performance. Johnson and White (1980) found 
that a minimal amount of training with a computer aided the 
performance of elderly subjects on a computerized test. Their 
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results suggest that with training it is possible to reduce the 
negative effects of testing with a computer. Their results also 
suggest that those who have more experience with computers are at 
an advantage when taking a computerized test. 

This study sought to investigate the effects of training with 
a computer and past computer experience on the computerized 
aptitude test performance of college students. Th<s study was 
conducted to determine (a) if computerized test performance would 
be significantly enhanced and the concomitant anxiety level would 
be significantly lowered by increasing the amount of training 
with the computer prior to testing and (b) if increasing the 
amount of training with the computer would significantly reduce 
the difference between the mean anxiety level occurring with a 
computerized tes^ and the mean anxiety level occurring with a 
paper-and-penci 1 test. Test performance was measured by che 
number-correct score on an arithmetic reasoning test. 



Me thods 

('*) Subjects 

Subjects were 92 undergraduates at the University of North 
Carolina at Charlotte and were given course credit for 
participation. There were 44 subjects (25 females and 19 males) 
in the Training Condition (Tr) and 48 subjects (29 females and ^9 
males) in the No Training Condition (NTr) . Five of the subjects 
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were black and 87 were white. The mean age was 21.59 years. 
Tests 

Arithmetic Reasoning Test. Two forms (ARPP and ARCOMP) of an 
arithmetic reasoning test were developed. Items were selected 
from a pool of word nroblems. The items on the two forms were 
matched judgmentally in terms of apparent difficulty and 
mathematical principies required. Each form contained 30 
multiple-choice items. The difficulty level of the tests was 
equivalent to that of high school math. The test required 
knowledge of basic algebra and geometry. One form of the test 
was completed by paper-and-penci \ (ARPP) . The other form of the 
test was completed by computer (ARCOMP). The number-correct 
score on the ARCOMP served as the dependent variable. 

Personal Preference Questionnaire . A 20-item Personal 
Preference Questionnaire (PPQ) was developed. The PPQ consisted 
of nonreactive multiple-choice questions similar to those used by 
Johnson and White (1980). The PPQ served as a practice tool for 
the subjects assigned to the Training Condition. 

Computer Experience Questionnaire . A 5-item Computer 
Experience Questionnaire (CEQ) was developed. For each item, 
subjects indicated the amount of experience they had with a 
computer in performing specified tasks. An example of an item 
is, "In the past year, how many letters or reports have you 
written using a word processor?" Options ranged from zero to more 
than three. Scores on the CEQ were used as a measure of past 
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computer training and experience. 

State -Trait Anxiety Inventory. The State scale of the 
State-Trait Anxiety Inventory (STAI; Spielberger, Gorsuch, & 
Lushene , 1970) measured anxie ty leve 1 . 
Appara tus 

Six Apple 11+ microcomputers were used to administer the 
ARCOMP. The microcomputers were housed in separa te rooms , each 
approKima tely 12 feet X 12 feet. The test questions were 
displayed singly on a cathode-ray tube (CRT), and the subjects 
typed the answers on a keyboard. Keys used to enter answers yjere 
specially labelled with bold, black letters on a white 
ba ckground. 
(5) Procedure 

Subjects were randomly assigned to the Tr and the NTr 
Conditions. All subjects completed two testing sessions. Each 
subject worked individually on each test. Time limits were not 
imposed and omitting of items was not allowed with any of the 
tests. 

Testing Session I. Subjects were tested in groups of two to 
seven. At the beginning of the session, subjects were told that 
the purpose of the study was to evaluate the arithmetic reasoning 
test and not the subjects as individuals. The test administrator 
remained in the room until all subjects had completed the tests. 
Each subject completed the ARPP then the STAI (STAI1), using 
paper-and-pencil for both. The ARPP was typed on standard typing 
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paper with 10 questions per page. The ARPP consisted of 
instructions for completing the ARPP, three sample questions, and 
the 30-item test. Answers for the ARPP were recorded on standard 
answer sheets to be read by an optical scanning machine. The 
ARPP served as an independent measure of a given subject's 
arithmetic reasoning ability and was a covariate in the data 
analysis. Standard forms for the STAI were used. 

Testing Session II. At least one week after Session I, each 
subject completed the following tests in the specified sequence: 
the STAI (STAI2), the PPQ, die ARCOMP , the STAI again (STAI3), 
and the CEQ. The test administrator did not remain in the room 
while the subject completed the ARCOMP. She was present only to 
give instructions and to begin the computer program. All 
subjects were told that the test administrator would be available 
if additional assistance was needed. 

Subjects in the NTr Condition completed all materials except 
the ARCOMP by paper-and-penc i 1 . In administering the .aRCOMP, the 
computer delivered (l) instructions on how to use the computer to 
indicate one f s answers, (2) the test instructions for the ARCOMP, 
(3) three sample questions, and (4) the 30-item ARCOMP. 

Subjects in the Tr Condition completed the PPO and the ARCOMP 
by computer; they completed the STAI2, the STAI3 , and the CEQ by 
paper-and-pencil. The Tr subjects practiced with the computer 
prior to completing the ARCOMP. The practice consisted of (1) 
completing the PPQ by computer and (2) feedback regarding one's 
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PPQ and sample question answers. Tr subjects were shown a paper 
computer printout of their answers to the PPQ and the three 
sample questions before they began the ARCOMP. The Tr subjects 
were also given additional verbal assurance by the test 
administrator that the computer would provide assistance and 
directions as needed. 

(6) Results 

No significant differences between the Tr and the NTr 
rubjects 1 ARCOMP scores were found. Nor was ARCOMP performance 
found co be significantly related to past computer experience (as 
measured by the CEQ). There was no significant difference 
between the anxiety levels of the Tr and the NTr subjects after 
the computerized test, as measured by STAI3. Mean anxiety level 
was significantly higher after the ARPP than after the ARCOMP, 
and the mean ARPP score was significantly higher than the mean 
ARCOMP score. 

Regression analysis was used to perform an analysis of 
covariance, with ARCOMP the dependent variable, ARPP a covariate, 
Amount of Training (Tr and NTr) the independent variable, and CEQ 
a measure of past computer experience. A significant interaction 
between ability (as measured by the ARPP) and Amount of Training 
was not found. The analysis showed that neither Amount of 
Training nor past computer experience accounted for a significant 
amount of ARCOMP variance, 

A separate analysis of ccvariance showed that anxiety level 
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after the computerized test was not significantly related to the 
Amount of Training or past computer experience. STAI3 was the 
dependent variable and STAI2 the covariate. The Amount of 
Training and computer experience were the independent variables. 

Since the Tr subjects and the NTr subjects were not 
significantly different in terms of STAI3 scores, all subjects 
were combined and a t-test was computed between anxiety level 
after the ARPP and anxiety level after the ARCOMP. Anxiety level 
was significantly higher after the ARPP (X = 38.1209 , SD = 
10.492) than after the ARCOMP (X = 34.8132, SD = 9.321), t(90) = 
3.3671 , p < .01), two-taiJed test. The mean ARPP score (X = 
19.696, SD = 6.270) was significantly higher than the mean ARCOMP 
score (X = 17.370, SD = 6.163), t(91) = 4.298, p < .01, 
two- tailed test. 

(7) Implications and Conclusions 
Neither training immediately before testing nor past computer 
experience appears to have significantly affected computerized 
test performance or the concomitant anxiety level. It appears 
that with college students, familiarity with the comupter is not 
an important factor in achieving optimal performance with a 
computerized arithmetic reasoning tests. These findings suggest 
that students' past equal access to computers may not be an 
important issue when testing young adults with computers. 
Computerized testing did not appear to discriminate against those 
who had less computer experience. These findings are contrary to 
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those of Johnson and White's study with elderly subjects. 
Perhaps for the college students at least a minimum amount of 
past computer experience was sufficient to equalize examinees. 
The mean CEQ score was 4,728 (Si) = 3.228). Only one subject had 
had no past computer expei'ience. 

Although this study was not designed to compare 
paper-and-penci 1 test performance with computerized test 
performance, it is interesting to note that the mean 
paper-and-penci 1 test score was significantly higher than the 
mean computerized test score. Moreover, not only was the mean 
ARPP score significantly higher than the mean ARCOMP score but 
the former was accompanied by a significantly higher anxiety 
level than the latter, indicating that the heightened anxiety 
level did not have a significant adverse effect on test 
performance. The ARPP and ARCOMP were developed to be as similar 
as possible in terns of content and difficulty. The matching of 
the items was done judgmen tal ly , but it is felt that close 
equivalence was achieved. The significant difference between the 
mean ARPP score arid the mean ARCOMP score is consistent with 
other studies (Lee et al. , 1984; Sachar & Fletcher, 1977). The 
current study focused on the effects of the amount of training 
and anxiety level on computerised test scores. Given the results 
of this study, it is felt that the differences found by some 
studies between paper-and-penci i test performance and 
computerized test performance may largely be due to human error 
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and deficiencies of the computer software. Some subjects in the 
current study remarked that some of the responses entered were 
done so by mistake. Responses to the computerized test could not 
be changed and subjects could not review past items. It is 
suggested that oniy software that allows the conveniences of 
paper-and-pencil tests, e. g. , the ability to change answers and 
the ability to review past items, be used in future applications. 
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